Joint NN-Supported Multichannel Reduction of Acoustic Echo, Reverberation and Noise

We consider the problem of simultaneous reduction of acoustic echo, reverberation and noise. In real scenarios, these distortion sources may occur simultaneously and reducing them implies combining the corresponding distortion-specific filters. As these filters interact with each other, they must be jointly optimized. We propose to model the target and residual signals after linear echo cancellation and dereverberation using a multichannel Gaussian modeling framework and to jointly represent their spectra by means of a neural network. We develop an iterative block-coordinate ascent algorithm to update all the filters. We evaluate our system on real recordings of acoustic echo, reverberation and noise acquired with a smart speaker in various situations. The proposed approach outperforms in terms of overall distortion a cascade of the individual approaches and a joint reduction approach which does not rely on a spectral model of the target and residual signals

End-To-End Label Uncertainty Modeling for Speech-based Arousal Recognition Using Bayesian Neural Networks

Emotions are subjective constructs. Recent end-to-end speech emotion recognition systems are typically agnostic to the subjective nature of emotions, despite their state-of-the-art performance. In this work, we introduce an end-to-end Bayesian neural network architecture to capture the inherent subjectivity in the arousal dimension of emotional expressions. To the best of our knowledge, this work is the first to use Bayesian neural networks for speech emotion recognition. At training, the network learns a distribution of weights to capture the inherent uncertainty related to subjective arousal annotations. To this end, we introduce a loss term that enables the model to be explicitly trained on a distribution of annotations, rather than training them exclusively on mean or gold-standard labels. We evaluate the proposed approach on the AVEC'16 dataset. Qualitative and quantitative analysis of the results reveals that the proposed model can aptly capture the distribution of subjective arousal annotations, with state-of-the-art results in mean and standard deviation estimations for uncertainty modeling.Comment: This paper is submitted to INTERSPEECH 202

Multiple-input neural network-based residual echo suppression

International audienceA residual echo suppressor (RES) aims to suppress the residual echo in the output of an acoustic echo canceler (AEC). Spectral-based RES approaches typically estimate the magnitude spectra of the near-end speech and the residual echo from a single input, that is either the far-end speech or the echo computed by the AEC, and derive the RES filter coefficients accordingly. These single inputs do not always suffice to discriminate the near-end speech from the remaining echo. In this paper, we propose a neural network-based approach that directly estimates the RES filter coefficients from multiple inputs, including the AEC output, the far-end speech, and/or the echo computed by the AEC. We evaluate our system on real recordings of acoustic echo and near-end speech acquired in various situations with a smart speaker. We compare it to two single-input spectral-based approaches in terms of echo reduction and near-end speech distortion

Joint NN-Supported Multichannel Reduction of Acoustic Echo, Reverberation and Noise: Supporting Document

This technical report is the supporting document of our proposed approach basedon a neural network for joint multichannel reduction of echo, reverberation and noise [1]. First, werecall the model of the proposed approach. Secondly, we express the vectorized computation of echocancellation and dereverberation. Thirdly, we detail the complete derivation of the update rules.Fourthly we describe the computation of the ground truth targets for the neural network usedin our approach. Fifthly we detail the variant of the proposed approach where echo cancellationand dereverberation are performed in parallel. Sixthly we describe the variant of the proposedapproach where only echo cancellation is performed. Then we specify the recording and simulationparameters of the dataset, we detail the computation of the estimated early near-end componentsand we recall the baselines. Finally we give the results after each filtering step and providesestimated spectrogram examples by all the approaches

Base de datos de abejas ibéricas

Las abejas son un grupo extremadamente diverso con más de 1000 especies descritas en la península ibérica. Además, son excelentes polinizadores y aportan numerosos servicios ecosistémicos fundamentales para la mayoría de ecosistemas terrestres. Debido a los diversos cambios ambientales inducidos por el ser humano, existen evidencias del declive de algunas de sus poblaciones para ciertas especies. Sin embargo, conocemos muy poco del estado de conservación de la mayoría de especies y de muchas de ellas ignoramos cuál es su distribución en la península ibérica. En este trabajo presentamos un esfuerzo colaborativo para crear una base de datos de ocurrencias de abejas que abarca la península ibérica e islas Baleares que permitirá resolver cuestiones como la distribución de las diferentes especies, preferencia de hábitat, fenología o tendencias históricas. En su versión actual, esta base de datos contiene un total de 87 684 registros de 923 especies recolectados entre 1830 y 2022, de los cuales un 87% presentan información georreferenciada. Para cada registro se incluye información relativa a la localidad de muestreo (89%), identificador y colector de la especie (64%), fecha de captura (54%) y planta donde se recolectó (20%). Creemos que esta base de datos es el punto de partida para conocer y conservar mejor la biodiversidad de abejas en la península ibérica e Islas Baleares. Se puede acceder a estos datos a través del siguiente enlace permanente: https://doi.org/10.5281/zenodo.6354502ABSTRACT: Bees are a diverse group with more than 1000 species known from the Iberian Peninsula. They have increasingly received special attention due to their important role as pollinators and providers of ecosystem services. In addition, various rapid human-induced environmental changes are leading to the decline of some of its populations. However, we know very little about the conservation status of most species and for many species, we hardly know their true distributions across the Iberian Peninsula. Here, we present a collaborative effort to collate and curate a database of Iberian bee occurrences to answer questions about their distribution, habitat preference, phenology, or historical trends. In total we have accumulated 87 684 records from the Iberian Peninsula and the Balearic Islands of 923 different species with 87% of georeferenced records collected between 1830 and 2022. In addition, each record has associated information such as the sampling location (89%), collector and person who identified the species (64%), date of the capture (54%) and plant species where the bees were captured (20%). We believe that this database is the starting point to better understand and conserve bee biodiversity in the Iberian Peninsula. It can be accessed at: https://doi.org/10.5281/zenodo.6354502Esta base de datos se ha realizado con la ayuda de los proyectos EUCLIPO (Fundação para a Ciência e a Tecnologia, LISBOA-01-0145-FEDER-028360/EUCLIPO) y SAFEGUARD (ref. 101003476 H2020 -SFS-2019-2).info:eu-repo/semantics/publishedVersio

Mortality from gastrointestinal congenital anomalies at 264 hospitals in 74 low-income, middle-income, and high-income countries: a multicentre, international, prospective cohort study

Summary Background Congenital anomalies are the fifth leading cause of mortality in children younger than 5 years globally. Many gastrointestinal congenital anomalies are fatal without timely access to neonatal surgical care, but few studies have been done on these conditions in low-income and middle-income countries (LMICs). We compared outcomes of the seven most common gastrointestinal congenital anomalies in low-income, middle-income, and high-income countries globally, and identified factors associated with mortality. Methods We did a multicentre, international prospective cohort study of patients younger than 16 years, presenting to hospital for the first time with oesophageal atresia, congenital diaphragmatic hernia, intestinal atresia, gastroschisis, exomphalos, anorectal malformation, and Hirschsprung’s disease. Recruitment was of consecutive patients for a minimum of 1 month between October, 2018, and April, 2019. We collected data on patient demographics, clinical status, interventions, and outcomes using the REDCap platform. Patients were followed up for 30 days after primary intervention, or 30 days after admission if they did not receive an intervention. The primary outcome was all-cause, in-hospital mortality for all conditions combined and each condition individually, stratified by country income status. We did a complete case analysis. Findings We included 3849 patients with 3975 study conditions (560 with oesophageal atresia, 448 with congenital diaphragmatic hernia, 681 with intestinal atresia, 453 with gastroschisis, 325 with exomphalos, 991 with anorectal malformation, and 517 with Hirschsprung’s disease) from 264 hospitals (89 in high-income countries, 166 in middleincome countries, and nine in low-income countries) in 74 countries. Of the 3849 patients, 2231 (58·0%) were male. Median gestational age at birth was 38 weeks (IQR 36–39) and median bodyweight at presentation was 2·8 kg (2·3–3·3). Mortality among all patients was 37 (39·8%) of 93 in low-income countries, 583 (20·4%) of 2860 in middle-income countries, and 50 (5·6%) of 896 in high-income countries (p<0·0001 between all country income groups). Gastroschisis had the greatest difference in mortality between country income strata (nine [90·0%] of ten in lowincome countries, 97 [31·9%] of 304 in middle-income countries, and two [1·4%] of 139 in high-income countries; p≤0·0001 between all country income groups). Factors significantly associated with higher mortality for all patients combined included country income status (low-income vs high-income countries, risk ratio 2·78 [95% CI 1·88–4·11], p<0·0001; middle-income vs high-income countries, 2·11 [1·59–2·79], p<0·0001), sepsis at presentation (1·20 [1·04–1·40], p=0·016), higher American Society of Anesthesiologists (ASA) score at primary intervention (ASA 4–5 vs ASA 1–2, 1·82 [1·40–2·35], p<0·0001; ASA 3 vs ASA 1–2, 1·58, [1·30–1·92], p<0·0001]), surgical safety checklist not used (1·39 [1·02–1·90], p=0·035), and ventilation or parenteral nutrition unavailable when needed (ventilation 1·96, [1·41–2·71], p=0·0001; parenteral nutrition 1·35, [1·05–1·74], p=0·018). Administration of parenteral nutrition (0·61, [0·47–0·79], p=0·0002) and use of a peripherally inserted central catheter (0·65 [0·50–0·86], p=0·0024) or percutaneous central line (0·69 [0·48–1·00], p=0·049) were associated with lower mortality. Interpretation Unacceptable differences in mortality exist for gastrointestinal congenital anomalies between lowincome, middle-income, and high-income countries. Improving access to quality neonatal surgical care in LMICs will be vital to achieve Sustainable Development Goal 3.2 of ending preventable deaths in neonates and children younger than 5 years by 2030

Apprentissage profond bout-en-bout pour le rehaussement de la parole

This PhD falls within the development of hands-free telecommunication systems, more specifically smart speakers in domestic environments. The user interacts with another speaker at a far-end point and can be typically a few meters away from this kind of system. The microphones are likely to capture sounds of the environment which are added to the user's voice, such background noise, acoustic echo and reverberation. These types of distortion degrade speech quality, intelligibility and listening comfort for the far-end speaker, and must be reduced. Filtering methods can reduce individually each of these types of distortion. Reducing all of them implies combining the corresponding filtering methods. As these methods interact with each other which can deteriorate the user's speech, they must be jointly optimized. First of all, we introduce an acoustic echo reduction approach which combines an echo cancellation filter with a residual echo postfilter designed to adapt to the echo cancellation filter. To do so, we propose to estimate the postfilter coefficients using the short term spectra of multiple known signals, including the output of the echo cancellation filter, as inputs to a neural network. We show that this approach improves the performance and the robustness of the postfilter in terms of echo reduction, while limiting speech degradation, on several scenarios in real conditions. Secondly, we describe a joint approach for multichannel reduction of echo, reverberation and noise. We propose to simultaneously model the target speech and undesired residual signals after echo cancellation and dereveberation in a probabilistic framework, and to jointly represent their short-term spectra by means of a recurrent neural network. We develop a block-coordinate ascent algorithm to update the echo cancellation and dereverberation filters, as well as the postfilter that reduces the undesired residual signals. We evaluate our approach on real recordings in different conditions. We show that it improves speech quality and reduction of echo, reverberation and noise compared to a cascade of individual filtering methods and another joint reduction approach. Finally, we present an online version of our approach which is suitable for time-varying acoustic conditions. We evaluate the perceptual quality achieved on real examples where the user moves during the conversation.Cette thèse s'insère dans le développement des systèmes de télécommunication mains-libres, en particulier avec des enceintes intelligentes en environnement domestique. L'utilisateur interagit avec un correspondant distant en étant généralement situé à quelques mètres de ce type de système. Les microphones sont susceptibles de capter des sons de l'environnement qui se mêlent à la voix de l'utilisateur, comme le bruit ambiant, l'écho acoustique et la réverbération. Ces types de distorsions peuvent gêner fortement l'écoute et la compréhension de la conversation par le correspondant distant, et il est donc nécessaire de les réduire. Des méthodes de filtrage existent pour réduire individuellement chacun de ces types de distorsion sonore, et leur réduction simultanée implique de combiner ces méthodes. Toutefois, celles-ci interagissent entre elles, et leurs interactions peuvent dégrader de la voix de l'utilisateur. Il est donc nécessaire d'optimiser conjointement ces méthodes. En premier lieu, nous présentons une approche de réduction de l'écho acoustique combinant un filtre d'annulation d'écho avec un post-filtre de suppression d'écho résiduel conçu de manière à s'adapter à différents modes de fonctionnement du filtre d'annulation. Pour cela, nous proposons d'estimer les coefficients du post-filtre en utilisant les spectres à court terme de plusieurs signaux observés, dont le signal estimé par le filtre d'annulation, en entrée d'un réseau de neurones. Nous montrons que cette approche améliore la performance et la robustesse du post-filtre en matière de réduction d'écho, tout en limitant la dégradation de la parole de l'utilisateur, sur plusieurs scénarios dans des conditions réelles. En second lieu, nous décrivons une approche conjointe de réduction multicanale de l'écho, de la réverbération et du bruit. Nous proposons de modéliser simultanément la parole cible et les signaux résiduels après annulation d'écho et déréverbération dans un cadre probabiliste et de représenter conjointement leurs spectres à court terme à l'aide d'un réseau de neurones récurrent. Nous intégrons cette modélisation dans un algorithme de montée par blocs de coordonnées pour mettre à jour les filtres d'annulation d'écho et de déréverbération, ainsi que le post-filtre de suppression des signaux résiduels. Nous évaluons notre approche sur des enregistrements réels dans différentes conditions. Nous montrons qu'elle améliore la qualité de la parole ainsi que la réduction de l'écho, de la réverbération et du bruit, par rapport à une approche optimisant séparément les méthodes de filtrage et une autre approche de réduction conjointe. En dernier lieu, nous formulons une version en ligne de notre approche adaptée aux situations où les conditions acoustiques varient dans le temps. Nous évaluons la qualité perceptuelle sur des exemples réels où l'utilisateur se déplace durant la conversation

End-to-end deep learning for speech enhancement

Cette thèse s'insère dans le développement des systèmes de télécommunication mains-libres, en particulier avec des enceintes intelligentes en environnement domestique. L'utilisateur interagit avec un correspondant distant en étant généralement situé à quelques mètres de ce type de système. Les microphones sont susceptibles de capter des sons de l'environnement qui se mêlent à la voix de l'utilisateur, comme le bruit ambiant, l'écho acoustique et la réverbération. Ces types de distorsions peuvent gêner fortement l'écoute et la compréhension de la conversation par le correspondant distant, et il est donc nécessaire de les réduire. Des méthodes de filtrage existent pour réduire individuellement chacun de ces types de distorsion sonore, et leur réduction simultanée implique de combiner ces méthodes. Toutefois, celles-ci interagissent entre elles, et leurs interactions peuvent dégrader de la voix de l'utilisateur. Il est donc nécessaire d'optimiser conjointement ces méthodes. En premier lieu, nous présentons une approche de réduction de l'écho acoustique combinant un filtre d'annulation d'écho avec un post-filtre de suppression d'écho résiduel conçu de manière à s'adapter à différents modes de fonctionnement du filtre d'annulation. Pour cela, nous proposons d'estimer les coefficients du post-filtre en utilisant les spectres à court terme de plusieurs signaux observés, dont le signal estimé par le filtre d'annulation, en entrée d'un réseau de neurones. Nous montrons que cette approche améliore la performance et la robustesse du post-filtre en matière de réduction d'écho, tout en limitant la dégradation de la parole de l'utilisateur, sur plusieurs scénarios dans des conditions réelles. En second lieu, nous décrivons une approche conjointe de réduction multicanale de l'écho, de la réverbération et du bruit. Nous proposons de modéliser simultanément la parole cible et les signaux résiduels après annulation d'écho et déréverbération dans un cadre probabiliste et de représenter conjointement leurs spectres à court terme à l'aide d'un réseau de neurones récurrent. Nous intégrons cette modélisation dans un algorithme de montée par blocs de coordonnées pour mettre à jour les filtres d'annulation d'écho et de déréverbération, ainsi que le post-filtre de suppression des signaux résiduels. Nous évaluons notre approche sur des enregistrements réels dans différentes conditions. Nous montrons qu'elle améliore la qualité de la parole ainsi que la réduction de l'écho, de la réverbération et du bruit, par rapport à une approche optimisant séparément les méthodes de filtrage et une autre approche de réduction conjointe. En dernier lieu, nous formulons une version en ligne de notre approche adaptée aux situations où les conditions acoustiques varient dans le temps. Nous évaluons la qualité perceptuelle sur des exemples réels où l'utilisateur se déplace durant la conversation.This PhD falls within the development of hands-free telecommunication systems, more specifically smart speakers in domestic environments. The user interacts with another speaker at a far-end point and can be typically a few meters away from this kind of system. The microphones are likely to capture sounds of the environment which are added to the user's voice, such background noise, acoustic echo and reverberation. These types of distortion degrade speech quality, intelligibility and listening comfort for the far-end speaker, and must be reduced. Filtering methods can reduce individually each of these types of distortion. Reducing all of them implies combining the corresponding filtering methods. As these methods interact with each other which can deteriorate the user's speech, they must be jointly optimized. First of all, we introduce an acoustic echo reduction approach which combines an echo cancellation filter with a residual echo postfilter designed to adapt to the echo cancellation filter. To do so, we propose to estimate the postfilter coefficients using the short term spectra of multiple known signals, including the output of the echo cancellation filter, as inputs to a neural network. We show that this approach improves the performance and the robustness of the postfilter in terms of echo reduction, while limiting speech degradation, on several scenarios in real conditions. Secondly, we describe a joint approach for multichannel reduction of echo, reverberation and noise. We propose to simultaneously model the target speech and undesired residual signals after echo cancellation and dereveberation in a probabilistic framework, and to jointly represent their short-term spectra by means of a recurrent neural network. We develop a block-coordinate ascent algorithm to update the echo cancellation and dereverberation filters, as well as the postfilter that reduces the undesired residual signals. We evaluate our approach on real recordings in different conditions. We show that it improves speech quality and reduction of echo, reverberation and noise compared to a cascade of individual filtering methods and another joint reduction approach. Finally, we present an online version of our approach which is suitable for time-varying acoustic conditions. We evaluate the perceptual quality achieved on real examples where the user moves during the conversation